Meta open-sourced Muse Glimmer, a 30-billion-parameter model designed to run on consumer GPUs and laptops for local AI agent tasks like scheduling, coding, and tool use. The model uses quantization and speculative decoding to fit within 20-32GB of memory while maintaining speed for real-time interaction. It's available now under Apache 2.0 license with integrations for llama.cpp, MLX, and other frameworks.

• Meta open-sourced Muse Glimmer, a 30B-parameter agentic model quantized to under 20GB so it runs locally on consumer GPUs/laptops (24-32GB) while competing with Gemma 4-31B and Qwen 3.6-27B.
• Speculative decoding with a lightweight drafter model proposes token blocks at once instead of one token at a time, speeding up reasoning and tool calls without changing outputs.
• Training combined logit distillation from a larger teacher model, agent-heavy mid-training data, and post-training RL/distillation across reasoning, coding, and agentic tasks.
• Released under Apache 2.0 with upcoming llama.cpp, MLX, and ExecuTorch integrations and support from Ollama, LM Studio, Together AI, and chipmakers like NVIDIA, Intel, AMD, and Arm.

This guide shows how to run Z.ai’s open-source GLM-5.2 model on local hardware using Unsloth Dynamic GGUF quantizations. It covers memory requirements for 1-bit and 2-bit setups, recommended inference settings, and step-by-step instructions for Unsloth Studio and llama.cpp. The article also explains KLD benchmarks and quantization accuracy trade-offs.

• GLM-5.2's 744B-parameter model can run on a 24GB GPU plus 256GB RAM using Unsloth's 2-bit dynamic quant (239GB, ~82% top-1 accuracy), an 84% size cut from the full 1.5TB model.
• The 1-bit quant shrinks total memory needs to ~223GB but drops accuracy to ~76.2%.
• 4-bit and 5-bit dynamic quants are nearly lossless per KLD testing, making them the better choice when accuracy on out-of-distribution tasks matters most.
• Dynamic quantization keeps critical layers at higher precision while aggressively compressing less important ones, which is why even 1-2 bit versions stay broadly usable despite the accuracy trade-off.